Comments for MEDB 5510, Week 06

Topics to be covered

What you will learn, part 1
- Secondary data analysis
- Centers for Disease Control and Prevention
- Bureau of the Census
- Cohort designs
- Case-control designs
- Cross-sectional designs
- Historical control designs
- Establishing causation in an observational design

What you will learn, part 2
- Qualitative data, phenomenology
- Grounded theory
- Ethnography
- The case sudy
- Narrative
- Community-Based Participatory Research

Learning objectives

To distinguish different types of quantitative non-experimental approaches
To discuss strengths and weaknesses of qualitative research

What is secondary data analysis?

Primary: data that you both collect and analyze.
Secondary: you analyze someone else’s data
- “Secondary analysis of existing data” (Cheng 2014)
Private versus government sources
Ancillary studies

There’s a nice paper on the recommended reading list that discusses secondary data analysis. Primary data is data that you (meaning you and your research team) both collect and analyze. Secondary dta is data that someone else collected and you only provide an analysis.

There’s an excellent reference in the recommended readings section and they emphasize that you should use the full phrase “secondary analysis of existing data” to avoid confusion.

Secondary data can come from private sources or from government sources. I’ll talk in a bit about two excellent government sources of data: the U.S. Census Bureau and the Centers for Disease Control and Prevention.

When you work with an existing research study and ask them to add a few items to collect for your benefit, you are working on an ancillary study. This is a special treat and requires close work with the team on the existing research study. In most secondary data analyses, you con’t have this luxury and have to take the data as is.

An ancillary study requires IRB approval. You need to discuss with the coordinators of the study that you are piggy baking on about who gets to be an author on your study, and what the order of authors should be. Do this before the paper is written, or you will find yourself in the middle of an ugly argument.

What can you analyze for an already analyzed data set?

Secondary data analysis is like eating left-over food.
“Think different”
- Variables that are analyzed
- Relationships that are explored
- Different subsets of cases
- Different analysis techniques

If the data set has already been analyzed, is there really anything else you can do? Well, yes, but you have to live with the fact that the researchers have already selected and analyzed the most interesting parts of the data. So you are like the last person in line at a pot-luck buffet. The Swedish meatballs and the apple cobbler are already gone and you’re left with the kale salad and the creamed spinach.

Don’t fret, though. Many researchers collect more data than what they strictly need for their own purposes.

You do need, as the Apple slogan says, to “Think different.”

You might look at different outcomes or different exposure variables that address the same general hypotheses. You could develop hypotheses involving new and untested relationships. You might be interested in a particular demographic subgroup. Or finally, you might test the same things that original researchers tested using more or less the same variables, but you might apply a new and different analysis approach.

Combining data sets

Third “V” in volume, velocity, and variety
Not the same as systematic overview, think “mash-up”
Examples
- Black box warnings and prescription use
- Staffing levels and patient complaints
- Pediatric asthma visits and housing survey

The combination of disparate data sets is one of the hallmarks of big data (the “variety” V of the three V’s of big data). This is not the same as a systematic overvew, which combines several stuides of the same hypothesis. Think of the term “mash-up” which is a combination of seemingly disparite elements to create a new, but coherent product.

Here’s a hypothetical example. The FDA provides lists of drugs that get “black box” warnings. This is a fairly serious thing, and you might be interested in seeing whether the prescription rates (as measured through the electronic health record) changed after the black box warning appeared.

Another hypothetical example is looking at staffing levels and the use of unplanned leave, which comes from personnel records, and the number and type of patient complaints, which are tracked in a complaints database.

I’ve worked with researchers who are looking at pediatric asthma visits at a local hopsital and linking the addresses to a housing survey conducted by the local government. The concern is that substandard housing could be associated with a greater need for asthma care.

Disadvantages of secondary analysis

Missing data
- Variables not collected
- Specific details not collected
Wrong data
- Wrong time
- Wrong measures
Stale data
Unable to fix obvious errors

One of the big ones is the lack of control over the data collection process. You can’t change how the data was collected without access to a time machine.

You may want infroation on certain variables, but the primary researchers chose not to collect this information. Or they did collect the information, but not at a level of detail that you want.

Maybe they collected the data, but collected it at the wrong time, or used a measurement approach that you dislike.

Because secondary analysis is retrospective, the data may lack “currency”. It may describe an health care setting that is so different from what is done today that it represents an inappropriate extrapolation.

In a prospective study, if you get an unusual result, you can ask your research team to double check things, and maybe even go back to the patient and get a fresh set of numbers. You need to do this carefully, but it is a nice option that is totally unavailable to you in a secondary data analysis.

Advantages of secondary analysis

Time
Money
Support

Two types of secondary data sets

Individual
- More analysis options
- Privacy concerns
Aggregate
- Mixture of apples and oranges
- Problems with the ecologic fallacy

Some secondary data sets that have individual level records: there is separate records an indvidual unit (usually a person, but it could be an event like an emergency department visit). This is preferable in most settings, but sometimes important information has to be redacted because of privacy concerns. Other pieces of information have random noise added to them to preserve confidentiality (e.g., data shifting).

Aggregate data avoids many of the privacy issues. Aggregate data is also easier to store and to present. One big problem is that when you aggregate across a category, you end up mixing apples and oranges together and this often destroys signals that might be apparent when you look at individual values. When you mix a bunch of indvidual colors together, the results is always a blah shade of gray.

Even if you do find a signal at the aggregate level, you have to be careful. What you see may be an artefact of the ecologic fallacy. The ecologic fallacy is the belief that an association discovered at an aggregate level will also hold at an individual level.

An example of the ecologic fallacy: Sales of cigarettes are greater in states that have higher suicide rates. You don’t know whether an individual who smokes is more likely to commit suicide.

Getting started with secondary data analysis

Start with an existing database
Get familiar with the data
- What variables are there?
- What variables are not there?
- How are the variables coded?
- Identify potential pairs for associational studies
Make sure you are answering an interesting question

You can attack a secondary data analysis from one of two ends. In a research driven study, you pose a question and then search for secondary data sources that might answer your question. In a data driven study, you decide on a particular secondary data source and then decide what interesting questions this data set might help you answer.

For the latter, read the documentation obsessively. Learn what variables you have and don’t have. Make sure yuo are happy with the detail of the category levels for any categorical variable. Identify pairs of variables that you might want to explore the associations.

If you start with the database, you may end up answering a question that is easy to answer with that database, but it is to a question that no one really cares about. Get a second opinion about the interest and relevance of what you are studying.

There’s a joke about someone who is searching along a street for a lost set of keys and a passerby volunteers to help them look. The keep looking for a while and the passerby asks where do you think you dropped your keys. The person responds, over there in that alley. So the passerby asks, so why are you looking for it out here? The person replies, because the light is so much better out here.

Moral: You have to be practical about what you can accomplish, but the easy to run study is not necessarily the study you should run.

Alternative strategy for secondary analysis

Start with a research hypothesis
- List the inclusion/exclusion criteria
- List the relevant variables
Search databases to see if they are a good fit.
- Do they have the right patients?
- Where is your control group?
- Do they have the right variables?
- Are there restrictions on using the database?

You may instead want to start with your hypothesis. This seems a lot better. Starting with the data set is like putting the cart before the horse. But it is a lot more work and there is no guarantee that you will find the right data.

With your hypothesis, carefully list the inclusion and exclusion criteria. List the important variables: your outcome variable, your independent variables, and any covariates that you need to adjust for.

Then start looking at some candidate databases. Do they have the right patients. Especially make sure that the database provides you with a good control group. Maybe the data set only has sick people, and you want a healthy control group. There’s nothing worse than collecting a bunch of data and then finding out that you did not have a control group for comparison.

Get help from the literature

How has database been used previously?
What has already been answered?
What was their analysis plan?
Understand the limitations.

Break #1

What you have learned
- Secondary data analysis
What’s coming next
- Centers for Disease Control and Prevention

“The Nation’s Health Protection Agency”

Nationwide cross sectional surveys

National Health Interview Survey

National Health and Nutrition Examination Survey

Behavioral Risk Factor Surveillance System

National Death Index

Influenza Hospitalization Surveillance Network

FluView

Foodborne Diseases Active Surveillance Network

Active Bacterial Core Surveillance

Summary

CDC
- NHIS
- NHANES
- BRFSS
- NDI
- FluSurv-NET
- FluView
- FoodNet

Break #2

What you have learned
- Centers for Disease Control and Prevention
What’s coming next
- Bureau of the Census

United States Bureau of the Census

Article I, Section II of the U.S. Constitution

Integrated Public Use Microdata Series International Partnership

American Community Survey

Master Address File

Census blocks

Topologically Integrated Geographic Encoding and Referencing System

American Housing Survey

Current Population Survey

Consumer Expenditure Survey

National Crime Victimization Survey

Summary

Census
- ACS
- MAF
- Census blocks
- TIGER files
- AHS
- CPS
- CES
- NCVS

Break #3

What you have learned
- Bureau of the Census
What’s coming next
- Cohort designs

Research Approaches

Why not randomize everything?

Can’t randomize because…
- Impossible
- Impractical
- Unethical
- Strong patient preference
- Retrospective studies

If you have an attribute independent variable, as your book points out, you cannot randomize. When you compare demographic groups, those who belong to one demographic group are stuck in that group. You can change a person’s race or gender at all, and changing a person’s age group requires a time machine. You could, possibly, change a person’s socio-economic status, as the film “Trading Places” illustrates, but this requires far too much effort to be done on a large scale.

Last week, we talked about setting where randomization was impractical. In a busy setting like an emergency room, you usually can’t impose the complexities associated with randomization.

You can’t randomize when you are studying a harmful exposure. It is unethical to deliberately expose someone to something that might hurt them. There are a few limited exceptions, such as a harm that is minor and easily reversible.

You can’t randomize when patients and/or the treating physicians have a strong preference for (or against) a particular treatment. I talked last time about laproscopic surgery and how it was hard to convince patients to be randomly assigned to the alternative: open surgery, which leaves a large scar.

Finally, you can’t randomize a retrospective study, since the data has already been collected.

Your book uses three categories for research: could randomize and did randomize; could randomize, but didn’t (and sneers at this choice); and couldn’t randomize (because you were “stuck” with an attribute random variable). I think it is better to think about the non-randomized studies using the five criteria listed above.

Of course, a non-randomized study could decide against randomization for more than one of the above reasons. Suppose you are studying the effect of second-hand smoke exposure. You couldn’t randomly marry off half of your participants to smokers and half to non-smokers. It would be impractical, because you have to wait for several decades for the cancer to develop. It is unethical because you are deliberately exposing someone to something that is potentially harmful. You also have to deal with strong patient preference. Most of us are quite fussy about who we agree to marry.

Departures from Gliner et al

“Observational” instead of “non-experimental”
Categorize by sample selection
- Cohort
- Cross-sectional
- Case-control
- Historical control

What is a cohort design?

Cohort defined by exposure
Compared to unexposed controls
Prospective or retrospective

Disadvantages of cohort designs

Disadvantages
- Difficult for rare diseases, long latency
- Selection bias, confounding
Advantages
- Can examine multiple outcomes
- Easy to explain
- Well defined comparison group
- Adaptable to longitudinal analysis

Let’s talk about the disadvantages first.

If you are examining a rare outcome, you need a pretty large sample size. The rule of thumb is that you would like to see 25 to 50 events in the exposed and unexposed groups. If an outcome occurs rarely, like 0.1% of the time, then you’d need tens of thousands of patients to accumulate a sufficient number of events.

If you are looking at an outcome that has a long latency period (such as cancer), you might need to follow your patients for several decades. Who has that sort of luxury? A retrospective cohort might work here, but you’d better have a database that includes information across a very broad time span.

In a study where the patients and/or the doctors choose who gets what, you have an issue with selection bias. Also, you have no ability to insure covariate balance between the exposed and unexposed groups, so confounding is a serious issue.

Advantages of cohort designs

Advantages
- Can examine multiple outcomes
- Easy to explain
- Well defined comparison group
- Adaptable to longitudinal analysis

Break #4

What you have learned
- Cohort designs
What’s coming next
- Case-control designs

What is a case-control design?

Cases defined by outcome
Comparison to controls without the outcome
Always retrospective

In a case control design, the cases are defined by the outcome measure. Did someone have a certain disease or experience a certain bad event? This might represent a diagnosis of cancer, or a heart attack, or suicide attempt.

Look back at the history of that patient and see there is a common history that might identify risk factors that explain the event. You need a comparison group, though, to see if what is common in your cases is or is not common in your controls. A psychologist who was noting the problem with many research studies involving sexual abuse noted with a bit of sarcasm that 95% of all sexual abusers have milk in their refrigerators.

In a case control design, the comparison group is people who did not experience the outcome. You compare the cancer patients to patients who did not have cancer. You compare heart attack patients to patients who did not have heart attacks. You compare suicide attempts to people who did not try to kill themselves.

Disadvantages of the case-control design

Disadvantages
- Counter-intuitive appearance
- Difficulty in identifying good controls
- Confounding

The biggest complaint about the case-control design it just doesn’t feel right. The case-control design is, to most people (including me), very counter-intuitive nature. You’re selecting people on an outcome and then seeing what influences that outcome. There’s no sampling going on here, so it almost seems like cheating. It’s not cheating, and the theoretical foundation for the case-control design was established in the 1950s.

The other big, big problem for the case control study is finding good controls. You want to find someone just like your lung cancer cases in everything except the lung cancer. Or someone just like a heart attack patient minus the heart attack. Or someone just like the suicide attempt without the attempt.

There is no selection bias in a case-control study. Nobody elects to be a cancer patient, for example. But confounding is a huge issue. Suppose your cases for a cancer patient are people who show up at the same hospital with a broken arm. People with a fracture are almost certainly a different age, have a different set of activities, and so forth. Any of these might be an alternative explanation for why the risk factor is more commonly seen in the cases than in the controls.

Advantages of the case-control design

Advantages
- Can examine multiple risk factors
- Efficient for rare diseases
- Great starting point for mysterious outcomes

A lot of people sneer at the case-control design as the weakest type of observational study, and while it does have significant problems, the humble case-control design has significant advantages over many other types of research.

The case-control design is great if you want to examine a broad range of risk factors. You’re at a picnic and half of the people come down with food poisoning caused by listeria. Was it the potato salad? The baked beans? The apple streudel? At a big picnic, there might be hundreds of foods that could have caused the listeria outbreak.

That okay, though. Just find out what everyone ate and if there is a food that almost everyone with listeria had and that almost no one in the healthy control group had, there’s your culprit.

The case control design is very efficient for studying rare diseases. You put out the word in the community (“Calling all cases. Calling all cases.”) and get everyone in for an interview.

It’s also a great starting point. There is something weird happening out there and we don’t know what is causing it. The very first studies of AIDS, back when we didn’t know a lot about the disease, were case control studies.

Selecting controls in a case-control design

Selecting controls
- Hospital/clinic based
- Community
- Relative/friends
- Within the same cohort

Break #5

What you have learned
- Case-control designs
What’s coming next
- Cross-sectional designs

What is a cross-sectional design?

Single group
- No selection by exposure
- No selection by outcome
Can be prospective or retrospective

Disadvantages of cross-sectional designs

Disadvantages
- Confusion about temporal ordering
- Selection bias, confounding

Advantages of cross-sectional designs

Advantages
- Examine multiple risk factors, multiple outcomes
- Realistic setting

There are several advantages to a cross-sectional design.

Because you are not selecting a single exposure or a single outcome, you have a lot of flexibility to explore multiple exposures against multiple outcomes.

The cross-sectional design is also the most natural and intuitive setting for a research study. You can envision a sampling process that produces this study and in many settings, this sampling produces a realistic picture of the health care process. More realistic than the case control design certainly, and more realistic, possibly, than the cohort designs where the two groups have an artificial distinction between them. You could even argue that a cross-sectional design is more natural than a randomized control trial, because volunteer bias and the extra attention that are part and parcel of a randomized trial make it, at times, a poor representation of how health care is delivered.

Break #6

What you have learned
- Cross-sectional designs
What’s coming next
- Historical control designs

What are historical control designs

Controls separated by time
- Similar to the single group, pre-post measurement
Sometimes separated by space
- Similar to the two group, post measurement only
Often separated by both space and time

Disadvantages and advantages of historical control designs

Disadvantages
- Confounding
Advantages
- Cheap and easy
Settings with 100% morbidity or mortality

Confounding is an issue for any observational study, but with the other three types of studies, you can try to control for this with careful selection of the control group. You can also use matching or statistical adjustments and these often can help a lot.

The historical control study, in most people’s minds, is beyond repair. The separation in time and space pretty much guarantees confounding and there really are no remedies that you can apply during the design or during the analysis that will convince anyone that there isn’t an alternate explanation that is an equally or more convincing explanation for the results you have found.

The big advantage of the historical control design is time and money. Your historical control group is, by definition, already sitting there waiting for you to use it. In most cases, the historical control group comes cheap or even for free.

Everyone sneers at a historical control group and for once I might be tempted to sneer at it also. There is one exception, though. If an exposure is associated with near 100% morbidity or mortality, then a historical control group works just fine. You would be hard pressed to convince anyone to forgo a treatment in this setting, so why even try. The point, though, is that any improvement will be patently obvious in a disease that, left to its own, kills almost everyone that it touches.

Break #7

What you have learned
- Historical control designs
What’s coming next
- Establishing causation in an observational design

Causation and observational designs

Observational designs CAN establish a causal relationship
- Just requires more work
- Control for confounding
- Bring in external evidence
Hill’s nine criteria (strength, consistency, specificity, temporality, biological gradient, plausibility, experiment, analogy)

Your book states very strongly that only randomized trials can establish a cause and effect relationship. Quasi-experiments can sometimes suggest or hint at a cause and effect relationship. And non-experimental studies are pretty much incapable of anything.

This persective is held by many in the research community, but it is wrong, wrong, wrong.

Observational designs can establish a causal relationship. Let me repeat this, because it is important. Observational designs can establish a causal relationship.

You just have to work a bit harder at it. Confounding, for example, is a serious problem for all observational studies, but there are tools like matching that can control for confounding during the design phase, or propensity score analysis, that can control for confounding during the analysis phase. You don’t, necessarily have to resort to these approaches in a randomized trial (though they can’t hurt). The point is that pretty much any reasonable randomized trial can establish a causal relationship. But in observational studies, only the very best studies can establish a causal relationship.

The other thing about observational studies is that there is strength in numbers. It is usually not just one study that establishes a cause and effect relationship. It is a series of studies. So your current observational study has to rely on information external to that study because by itself, it may not be strong enough.

I want to review Sir Austin Bradford Hill’s criteria for establishing a causal relationship. He wrote a landmark paper in 1965 in the midst of the crisis in the research community about the link between smoking and lung cancer (more on that in a bit). He looked at the evidence that had accumulated and elucidated nine factors that you can use to help establish a causal relationship. None of these by themselves is sufficient, but a reasonable combination of them makes your argument quite persuasive.

Hill’s criteria, strength

Strength
- Large effects can only be overturned by large confounders.
- Weak effects can still be real

Hill identified the strength of the association as being the most critical factor. The reason for this is that a strong effect could only be overturned by a strong confounder. A weak confounder would only attenuate things a bit. If you want to hypothesize that a third factor among smokers (poor diet, dangerous work conditions, or lack of exercise) could cause the ten to twenty fold increase in the risk of lung cancer, that third factor would have to increase the risk of lung cancer by more ten or twenty fold.

Keep in mind, though, that weak effects can still be real. There was some criticism of a government report that said that the number of deaths due to smoking was around 450 thousand people per year. The critics dissected this number and said, we’ll allow you the deaths due to cancer, but not for other factors like stroke and heart attacks, because these only showed a two fold or smaller increase in risk. It turns out that these other ways that smoking can kill you are very real, but because these causes of death (unlike lung cancer) are fairly common in the general population, you can’t just get a large increase in risk. If 30% of the public dies from heart attacks, a ten fold increase is just impossible.

When the association is small, you don’t totally disregard it. You just have to rely on some of the other criteria.

The other point, that has been made in many settings is that a large effect can be easily produced by a seriously flawed study. Paul Rosenbaum, in his book, Observational Studies discusses a non-randomized study conducted by Linus Pauling that showed a four fold increase in the survival times in patients with advanced cancer who received mega-doses of Vitamin C, and then points out the serious flaw that caused this finding.

Hill’s criteria, consistency

Consistency
- Replication across DIFFERENT study types

Hill’s criteria, specificity

Specificity
- Multiple cures or common bias?
- Deliberate inclusion of negative outcomes
- Exceptions: aspirin, smoking

Specificity means that a treatment cures one and only one disease. If it appears to cure a multitude of diseases, then what is probably happening is that there is a bias (maybe selection bias?) that is causing the exposure group to be healthier than the control group. If an exposure is causing a multitude of problems, maybe a bias is causing the exposure group to be sicker than the control group.

You can rule out a healthier or sicker exposure group if you show that only one outcome (or only a small number of outcomes) is changed between the exposed and unexposed groups. You can take advantage of this by deliberately including outcome measures that you would expect to be negative.

Suppose you are testing whether toll booth workers suffer problems from CO poisoning. You’d get your toll booth workers and compare them to a control group and see if you have more problems with headache, nausea, dizziness, and fatigue. These are all symptoms associated with carbon monoxide poisoning. But what if the people who work in toll booths aren’t really sicker, but because they have such a dreary job, they are more likely to complain.

To check for this, ask about muscle pain, rashes, coughing, sneezing, fevers: symptoms that are not normally associated with CO poisoning. If these are elevated in the toll booth workers, then you know that you just have a bunch of complainers on your hands. If these show little or no change, while the symptoms specific to CO poisoning do show a change, then you have stronger evidence that something real is going on.

Hill’s criteria, temporality

Temporality
- A has to precede B to be a cause
- Advantage of prospective studies
- Difficult for long latency diseases

If you cannot establish temporality, then you have a harder time establishing causation.

This is easier in a prospective study. When you are recalling events in the past, it is easier to get them out of order. This is an issue I mentioned earlier with cross-sectional studies.

There were some speculations about how cancer causes smoking. This seems like a reversal of the causal model, but the argument was that some people have a genetic disposition to cancer and the same set of genes might also make them easier to get addicted. The argument doesn’t hold water, of course, but you need that genetic link to keep from being totally disregarded.

Temporality is tricky for a disease that has a long latency period, one that hides quietly in your body for many years before revealing itself. Did the disease change your diet rather than your diet producing the disease?

Hill’s criteria, biological gradient

Biological gradient
- Dose response relationship
- Rule out some, but not all confounders
- Hormesis and other patterns

A biological gradient means that as the intensity of your intervention increases, the larger the improvement you see in your outcome. For an exposure, a biological gradient means that the greater the amount of exposure, the greater the degree of harm.

The biological gradient is often referred to as a dose-response relationship. If you have the ability to measure several levels of an intervention or exposure, it can help. Some, but not all, biasing factors tend to bias at a constant rate, not at a rate that escalates in a dose-response pattern.

For example, recall bias is a big problem in retrospective studies. People’s memory of the past is influenced by what has happened to them recently. It’s not that one group is making things up, but rather that the controls are more likely to overlook things.

Recall bias, though, is unlikely to be stronger in patients with a greater exposure, so if you see a gradient, with low exposures showing smaller effects than large exposures, that’s good news. You can rule out recall bias and maybe selection bias and maybe even other biases if they affect your comparison more or less uniformly across the spectrum of exposures.

Hormesis is a surprising, but not too unusual pattern where you see a helpful effect for a small dose of something that is decidely harmful for larger doses. There is some evidence, for example, that a small amount of wine (e.g., one glass per day) has some beneficial effects, even though a larger amount of alcohol consumption causes all sorts of problems. So failure to see a dose response pattern is not a disqualifying event.

Hill’s criteria, plausibility

Plausibility
- Biological mechanism
- Dependent on current state of knowledge

Plausibility is a measure of whether a result is consistent with what we know about medicine. It is often referred to as a plausible biological mechanism.

It is commonly cited as a factor in deciding whether a result should be trusted, but in the original 1965 publication, Sir Austin Bradford Hill seems somewhat unimpressed. He points out, correctly, that our knowledge of mechanisms changes over time.

There is also a fair amount of subjectivity in this term. Just to cite one extreme example, there was a surge about a decade ago in prayer studies where patients were randomly assigned to either receive prayers from a stranger or to a control group. There is certainly a big difference in opinion about whether it is plausible to believe in a God who intercedes in our lives when we pray to Him.

The other problem with plausibility is that it is not too hard to come up with a biological mechanism for just about anything. So oral contraceptives could have an effect on the risk of breast cancer, but the mechanism could be an adverse effect because of the estrogen in oral contraceptives, or you could just as easily come up with a mechanism that is protective because of the progestin.

In spite of Hill’s reservations, a plausible mechanism is a great thing to have on your side. Lack of plausibility, however, could just mean that we haven’t discovered the mechanism yet.

Hill’s criteria, coherence

Coherence
- Follows natural history
- Consistent with biology

Hill’s criteria, experiment

Experiment

Sir Austin Bradford Hill developed his nine criteria for settings where direct experimental manipulation of the exposure would be impossible, but sometimes you can find an indirect way to run an experiment.

There are settings called natural experiments where a change in exposure was mandated by some fluke or accident. This is not as good as randomization, but the nature of the accident was unlikely to produce serious biases. This could be a law passed to ban import of pesticides, or a change in alcohol taxes. See the Craig 2012 article in the recommended readings.

You might also be able to run an experiment with random assignment not to a particular exposure but rather to an intervention that is hoped might reduce the exposure. So, you can’t randomly assign women to breast feeding versus bottle feeding, partly because it would be unethical and partly because of strong patient preference. But what you can do it to encourage every new mother to breast feed her child for six months. And then for a randomly selected subset of these patients, you would give them extra resources, such as access to a lactation consultant, free breast pumps, and regular visits by a home nurse. If those extra resources provided a significant change in breast feeding, then you would probably be able to measure improvements in the health and well-being of both the child and the mother.

Hill’s criteria, analogy

Analogy
- Similar to coherence?

Break #8

What you have learned
- Establishing causation in an observational design
What’s coming next
- Qualitative data, phenomenology

Five approaches to qualitative research

Qualitative – 5 main approaches
- Phenomenology
- Grounded theory
- Ethnography
- Case study
- Narrative
Relies on a constructivist philosophy
- Rejection of the single reality of positivism
- Research protocol adapts as new information emerges

Even if you never run a qualitative research study, you need to get an appreciation for how these studies are designed. They complement quantitative studies well and answer questions that would be difficult or impossible to answer with quantitative methods.

These approaches are all inductive (using specific findings to develop broader theories). You are trying to develop a framework of understanding. Don’t go in with a heavy set of expectations.

Qualitative research relies heavily on a construcivist philosophy. This philosophy is a rejection of positivism, the belief that there is a an objective truth and that there is an objective method to get at this truth. The research methodology is adaptive. How you collect information changes all along the way.

This sounds subjective, but the constructivist argues that you become more objective when you put aside your pre-conceived notions and assumptions.

I hope to talk more about this in week 12.

What is phenomenology?

Phenomenom: “an observable fact or event” (Merriam-Webster)
- What meaning do people place on this fact or event?
- How do people construct their own reality around certain events

An example of phenomenology, 1 of 2

An example of phenomenology, 2 of 2

Image of Pallesen 2019 abstract

Break #9

What you have learned
- Qualitative data, phenomenology
What’s coming next
- Grounded theory

What is grounded theory?

Generate theory from data collected from participants
- Inferences firmly “grounded” in the data
- Prior theoretical expectations avoided
- Sampling proceeds parallel to data collection & analysis
- Research maintains skepticism, seeks disconfirming examples

An example of grounded theory, 1 of 2

Break #10

What you have learned
- Grounded theory
What’s coming next
- Ethnography

What is ethnography?

Study of individuals who share the same culture
- Methods developed in Sociology/Anthropology
- Strong emphasis on observation

An example of ethnography, 1 of 2

Image of Khoei 2018 article

An example of ethnography, 2 of 2

Image of Khoei 2018 abstract

Break #11

What you have learned
- Ethnography
What’s coming next
- The case sudy

What is a case study?

In depth examination of a case or series of cases
- Not the same as a case report
- Viewed from a variety of lenses
Narrow definition
Emphasis on unusualness

An example of a case study, 1 of 2

Image of Phehlukwayo 2018 article

An example of a case study, 2 of 2

Image of Phehlukwayo 2018 abstract

Break #12

What you have learned
- The case sudy
What’s coming next
- Narrative

What is a narrative?

Narrative is a written account of an event
- Story developed from a variety of perspectives
- Use interviews, documents, artifacts

An example of a narrative, 1 of 2

Image of Nakanishi 2019 article

An example of a narrative, 2 of 2

Image of Nakanishi 2018 abstract

Break #13

What you have learned
- Narrative
What’s coming next
- Community-Based Participatory Research

Problems with traditional research

“Researchers are like mosquitoes; they suck your blood and leave” from Cochran 2008.
Harms of research
- Stigmatization
- Condescension
- Reinforcement of stereotypes
- Cultural insensitivity
- Failure to respect community standards
- Abandonment when the research is done
- Deception about true research purpose

I’m going to have some harsh words for you now, but you need to hear them. Researchers don’t always make the world a better place.

There is an increasing recognition that research often tends to benefit the person doing the research, and not the people who participate in the research. In fact, sometimes the research harms the participants, through stigmatization, condescending attitudes, and the reinforcement of harmful stereotypes. Researchers have abused their position by deceiving the research participants about the true goal of the research. They often fail to recognize and respects standards within the communities that they conduct their research in. There is also often a sense of abandonment, once the research is done, the participants never see or hear anything about the work.

This is not true about all research and all researchers. It isn’t even true about most research, I hope. But it does happen much more often than it should and it “poisons the well” and makes future research more difficult.

What is Community Based Participatory Research (CBPR)?

Designed in collaboration with the people being researched
Also known as Participatory Action Research (PAR)
Not restricted to any research methodology

Disadvantages of CBPR

Designed in collaboration with the people being researched
Also known as Participatory Action Research (PAR)
Not restricted to any particular research methodology

Advantages of CBPR

Advantages to you, the researcher
- Precisely targeted intervention
- Improved participation rate
- Greater generalizability
Advantages to the community
- Sustainability
- Greater community capacity

There are several big advantages to you, the researcher. By involving the participants in the research planning, you will be better able to target your intervention to the specific needs of this particular community. Second, the people in the community who help you plan the research will know better than you do how to design your study to make it attractive for the research participants. This will greatly improve your participation rate. The process of involving the community will almost always place the research in a more naturalistic context, which improves your generalizability.

There are advantages to the community beyond just making you, the researcher, less arrogant. When the community makes a commitment to the research process this insures a better level of sustainability.

The community based process helps the community grow and develop more capacity for research (and research that actually benefits them). They know as little about your world as you know about theirs. They, like you, will learn new skills. This will help the community in their interactions with future researchers.

Getting started with CBPR

Pick a problem
- Simple
- Important
- Needed
Ideal settings
- Health disparities
- Complex interventions
Identify resources
Keep an open mind
Be willing to compromise

If you are new to CBPR, start with something that is simple, important, and needed. Pick something that you know should add value to the community.

CBPR has been applied effectively in health disparities research because often you are already doing quite well in some communities, but there are barriers in other communities that keep you from delivering a uniformly high quality of care. The community can help you identify and overcome those barriers.

CBPR is also ideal for complex interventions, because the community can help you package the components that are most important from their perspective.

Identify people who can be resources. Keep an open mind about the process. There is little value in insisting on a rigid approach. Also understand that compromise will be necessary. You need to sacrifice some of the things that you want in order to get the cooperation of people who have different goals than you do.

Emphasize your strengths

Things you know that they don’t
- Existing research evidence
- Ability to diagnose
- Natural course and history of disease
- How to measure
- Research standards

Understand the strengths of the community members

Things that they know that you don’t
- Specific needs of their community
- Where to find research volunteers
- What they will and will not tolerate
- How to tweak an intervention
- Pragmatic advice

Summary

What you have learned, part 1
- Secondary data analysis
- Centers for Disease Control and Prevention
- Bureau of the Census
- Cohort designs
- Case-control designs
- Cross-sectional designs
- Historical control designs
- Establishing causation in an observational design

What you have learned, part 2
- Qualitative data, phenomenology
- Grounded theory
- Ethnography
- The case sudy
- Narrative
- Community-Based Participatory Research